Some Explicit Cases of the Selberg Trace Formula for Vector Valued Functions

The trace formula for SL{2,Z) can be developed for vector-valued functions which satisfy an automorphic condition involving a group representation n . This paper makes this version explicit for the class of representations which can be realized as representations of the finite group PSL(2,Z/q) for some prime q . The body of the paper is devoted to computing, for the singular representations n , the determinant of the scattering matrix <&{s,n) on which the applications depend. The first application is a version of the Roelcke-Selberg conjecture. This follows from known results once the scattering matrix is given. The study of representations of SL{2, Z) in finite-dimensional vector spaces of (scalar-valued) holomorphic forms dates back to Hecke. Similar problems can be studied for vector spaces of Maass wave forms, with fixed level q and eigenvalue X . One would like to decompose the natural representation of SL(2,Z) in this space, and count the multiplicities of its irreducible components. The eigenvalue estimate obtained for vector-valued forms is equivalent to an asymptotic count, as A —► oo , of these multiplicities. The trace formula for SL(2,Z) can be developed for vector-valued functions which satisfy an automorphic condition involving a group representation n . In fact, Selberg's original paper stated the trace formula in this generality. This paper makes this version explicit for the class of representations which can be realized as representations of the finite group PSL(2,Z/q) for some prime q . We then present some applications. The body of the paper is devoted to computing, for the singular representations n , the determinant of the scattering matrix (s , n) on which the applications depend. The Dirichlet series coefficients of <P(s, n) are matrix coefficient functions for n . To compute the scattering matrix one must first classify the singular representations and determine the matrix coefficients. By using the Frobenius reciprocity theorem, we show that the singular representations are essentially the principal series for the finite group, i.e., induced from characters on the subgroup of upper triangular matrices. Matrix coefficients for the principal series are easily computed. The scattering matrix is then summed by standard number theoretic techniques. The first application is a version of the Roelcke-Selberg conjecture; i.e., an estimate on the asymptotics of the eigenvalues for the Laplace operator in the relevant Received by the editors December 26, 1987 and, in revised form, April 25, 1988. 1980 Mathematics Subject Classification ( 1985 Revision). Primary 11F72.